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Abstract 



To estimate the effects of adaptive education on cognitive, affective, 
and behavioral outcomes of learning, 309 effect sizes were calculated using 
statistical data from 38 studies that contained a combined sample of 
approximately 7,200 students. The mibstantial mean of the study-weighted 
effect sizes is .45, suggesting that the average student in adaptive programs 
scores at the 67th percentile of control group distributions. The effects 
appear constant across grades, socioeconomic levels, races, private and public 
schools, and community types. In addition, the effects are not significantly 
different across the categories of adaptiveness , student outcomes, social 
contexts, and methodological rigor of the studies. 
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ADAPTIVE EDUCATION 
AND STUDENT OUTCOMES: 

A QUANTITATIVE SYNTHESIS 

Hersholt C. Waxman, Margaret C. Wang, 
Kenneth A. Anderson, and Herbert J. Walberg 

Although instruction that is adapted to individual differences has a long 
history (Whipple, 1925), it recently has been of great interest in current 
afforts aimed at improving educational opportunities for school-aged children 
and young adults (Birch & Reynolds, 1982; Fenstermacher & Goodlad, 1983). 
Executive, judicial, and legislative branches of government have called for 
increasing schools' responsiveness to the diverse needs of their students 
(National School Public Relations Association, 1981; Reynolds & Wang, 1981). 
School administrators and teachers indicated in a recent survey their 
preference for assessment of each student's capabilities and instruction that 
builds upon each student's skills (Rothrock, 1982). Furthermore, adapting 
instruction to the individual needs of the increasingly diverse student 
population in regular classes has come to be a major topic in effective 
schooling literature (Wang & Lindvall, 1984). 

Adaptive programs make use of a variety of curricula and techniques that 
have proven effective in many types of classroom settings and with a diversity 
of students. These include mastery learning, cooperative teams, and 
individual tutorials, as well as large- and small-group instructional 
approaches that have shown moderate to large effect sizes in previous research 
syntheses 'alberg, 1984). Adaptive programs are intended to make use of 
these techn jes in a manner best suited to each teacher, class, and student, 
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and there is considerable variety across programs. Within any program, 
moreover, teachers are expected to vary in their use of materials and 
procedures; and students, of course, are expected to be treated differentially 
according to their educational needs (see Wang & Lindvall, 1984). Adaptive 
educators contend that the use of such programs provides learning experiences 
that meet the needs of special and general education students in regular 
classroom settings and, therefore, results in opportunities for most students 
to succeed in their school learning. 

Despite increased interest in research and development activities aimed 
at ensuring success for each student, there are fiome inconsistent and 
difficult-to-interpret results on the educational impact of adaptive 
instruction. The present synthesis investigates those results by addressing 
the following questions: First, how extensive is the empirical evidence on 
the relationship between adaptive education and student outcomes? Second, 
what is the magnitude and direction of the relationship between adaptive 
education and student outcomes? Third, are there certain social contexts or 
student characteristics that affect the relationship? Fourth, are there 
pa. ticular methodological characteristics that affect the relationship? 
Finally, are there some specific characteristics of adaptive education that 
affect its relationship with student outcomes? 

To answer these questions, the present study quantitatively synthesized 
the experimental and quasi-experimental, published and unpublished research on 
the effects of adaptive education on student outcomes in naturalistic 
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settings. The techniques of research synthesis that were applied were derived 
from the work of Glass, McGaw, and Smith (1981) and Hunter, Schmidt, and 
Jackson (1982) on meta-analysis, as well as from contributions of Hauser-Cram 
(1983), Cooper and Rosenthal (1980), and Walberg and Haertel (1980). 

Method 

Search and Selection Procedure 

A systematic collection of a decade of research published from 1973 
through 1982 investigating the effects of adaptive education on student 
outcomes was conducted by searching Current Index to Journals in Education , 
Resources in Education , Review of Educational Research , American Educational 
Researc h Journal , Journal of Educational Psychology , Journal of Educational 
Research , and Journal of Experimental Education . Studies cited in the 
references listed in these sources also were examined. 

Several criteria were established for inclusion in the present synthesis. 
The study had to have been conducted in a regular classroom setting in a 
public or private elementary or secondary school. It ha<? to have either 
contrasted-groups or correlational results. In addition, the study had to 
have sufficient quantitative data bearing on the sample population to 
calculate effect sizes (Glass, McGaw, & Smith, 1981). An effort was made to 
include not only published articles but also dissertations and unpublished 
reports showing quantitative effects on student outcomes. 
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In order to be considered a study of adaptive education, and, therefore, 
be included in the synthesis, the program investigated had to contain at least 
one of the following characteristics (Wang & Lindvall, 1984): 

1. Instruction based on the assessed capabilities of each student. 

2. Materials and procedures that permit each student to make progress 
in mastering the curricula at a pace suited to his or her abilities 
and interests. 

3. Periodic evaluations of progress to inform the student of his or 
her mastery. 

4. Students' assumption of self-responsibility for diagnosing their 
current needs and abilities, for planning and pursuing individual 
learning activities, and for evaluating mastery. 

5. Alternative learning activities and materials for aiding students' 
acquisition of academic skills and content. 

6. Student choice in selecting educational goals, outcomes, and 
activities. 

7. Students' assistance to one another in pursuing individual goals 
and cooperation in achieving group goals. 



Tn those cases where the programs were not completely described in the 
article, additional sources were consulted to accurately bbu^bb whether any of 
these characteristics of adaptive education were present. 



Certain types of studies and reports were excluded from the synthesis. 
Special education populations (e.g., students classifiea as mentally retarded, 
learning-disabled) *ere left out. Reports of computer-assisted instructional 
programs and other types of programmed instruction also were omitted. Studies 
comparing several different instructional approaches (e.g., Becker, 1978; 
Bosco, Harring, & Bandy, 1976) were excluded, as were studies such as Slavin 
and Tanner (1979) and Johnson, Johnson, and Scott (1978), which compared 
various components of adaptive education (e.g., cooperative versus 
individualized instructional programs), because in such instances it was 
difficult to determine which program should be used as the control. In 
addition, only the most recent of multiple reports of the same population and 
results was included in order to maintain statistical independence. This may 
have occurred when the investigator conducted an empirical study to satisfy 
the requirements of a Ph.D. dissertation and subsequently published an article 
reporting the same results. 

The search and selection procedures resulted in a collection of 38 
studies. Of these, 24 were published articles, 10 were unpublished reports, 
and four were dissertations. 
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Analysis 



To calibrate the studies' results, or place the* on a common scale, 
"effect sizes" were calculated. These consisted of the treatment group Man 
minus the control mean divided by the control standard deviation. Effect 
sizes can be considered a standardized estimate of where the treatment group 
stands in comparison to the control group distribution. In the present case, 
a positive effect size indicated that the adaptive education group received 
higher (i.e., more desirable) scores than the control group. Glass, McGaw, 
and Smith's (1981) formulas were employed for studies that did not report 
group means or standard deviations but contained JF or Rvalues, correlations, 
or other statistics from which effect sizes could be calculated. 

For the present synthesis, one investigator recorded 39 codable 
characteristics and other data for each of the 309 effect sizes from 38 
studies. (The coding categories are listed in the Appendix.) A seconc 
investigator independently coded five studies. The intercoder agreement for 
each study reviewed exceeded the 85Z criterion. The methodological threats to 
validity were adopted from Cook and Campbell (1979). The 39 categorical 
variables were employed as factors in an analysis of variance (ANOVA). 

The studies varied by the number of comparisons they reported. 
Therefore, those studies with a greater number of comparisons (e.g., those 
that reported separate results by ability level, sex, or race) would have been 
weighted more heavily than others if each comparison had been given equal 
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weight. To give all studies the same unit weight in the analysis, each 
comparison was weighted in inverse proportion to the number of comparisons in 
the study from which it was taken (i.e, 1/n where n * number of comparisons in 
the study). Each of Goldner"f (1973) four comparisons, for example, received 
a weight of .25. 

Results and Discussion 

Characteristics of the 7,200 students in the 38 studies covered a broad 
spectrum. The median number of students in the sample of studies was 108 and 
the range was from 19 to 3,018 students. Sixteen studies involved primary 
students (grades K-3), 11 included intermediate ^udents (grades 4-6), and 11 
focused on junior-high and high- school students. Eleven studies focused on 
math as the subject area and 10 studies involved reading. Language arts and 
science were each the focus of four studies, while the remaining studies 
included a variety of other subject areas. 

The 309 comparisons are displayed in a stem-and-leaf diagram (Tukey, 

1977) in Figure 1. The stem-and-leaf diagram is useful for exhibiting such 

results because it provides all the information of a histogram as well as 

displaying the actual value of each effect size to two decimal places. The 

first digit, or stem, appears to the left of the vertical line, and subsequent 

digit(8), or leaves, appear to the right. For example, the leaves 0, 1, and 2 

to the right of the stem 1.5 represent three effect sizes: 1.50, 1.51, and 

1.52. Figure 1 also rnports the five-point summary of the distribution, 

consisting of minimum (*), first quartile or hinge (H), median (M), third 

quartile or hi f nge (H) , and maximum (*). 
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Figure 1. Stertvand-leaf and five-point summary of effect sizes. 
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Figure 1 shows that the majority of comparisons lie between -.4 and 1.5, 
but that the range extends from -2.00 to 3*50. Fifty percent are between .02 
and .50. Seventy-seven percent of the effect sizes are positive and favor the 
adaptive education groups. 

The 38 study-weighted effect sizes between adaptive education and student 
outcomes are displayed in Figure 2. The majority of the study-weighted effect 
sizes lie between .1 and .7. The range extendi from -1.18 to 1.73. 
Thirty-four of the 38 comparisons, or 89Z, are positive. The binomial 
probabilities of the evidence, given the hypothesis of an even 50-50 or chance 
split, is less than .01 (Dixon & Massey, 1957, p. 417). 

Table 1 lists the 38 studies synthesized, the number of comparisons in 
each study, the mean study-weighted effect size for each study, and a 
breakdown of the mean study-weighted effect size for each student outcome. 
Effect sizes were converted to percentiles by obtaining the item in the column 
of percentiles corresponding to the Z-score equal to the effect aize in a 
standard table of the normal distribution (see, for example, Talmage, 1976, 
pp. 60-61). The mean study-weighted effect size for all 30> comparisons for 
the 7,200 students is .45. This suggests percentiles on student outcomes of 
67 and 50 for the experimental and control groups respectively. The mean 
study-weighted effect size for the 208 comparisons containing cognitive 
outcomes is .39, suggesting percentiles on student cognitive outcomes of 65 
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Figure 2. Stem-and-leaf and five-point summary of study -weighted effect sizes between adaptive education 
and student outcomes. 
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Table 1 
Efftct Sizes for Etch Study 



Means by Outcom t 

Stud V N Study Mean S.D. Cognitive Afftctivt Bthavioral 



Abel son (19/4) 


44 


.144 


.384 


.201 


-.113 


Abrams tt al. (1979) 


1 


.190 




.190 




Bar-Tai et al. (1978) 


9 


1.210 


1.240 




1.210 


oreuning (1978) 


3 


.119 


.430 


.119 




Broussard \ i9/o) 


1 


.536 




.536 




P*mnk*ll /1Q7Q\ 

uampDen iiy/o) 


1 


.572 




.572 




uiasen tiy/H/ 


8 


.543 


.504 


.543 




p A w An 9. Di.Jmi.iim / « nort \ 

uonen « Hodnquez (19o0) 


2 


.528 


.063 


.528 




oonoy « Lewis (iy/bj 


9 


.350 


.446 


.350 






l 


.569 




.569 




p*nnell (1973) 

1 Jill IV II | Iwf w( 


0 
0 


1 7*5 
.1 /<£ 


.206 


.172 




Fesleretal (1976) 


Q 


.000 


ono 
.oQZ 


.685 




Goldntr (1973) 


A 

*T 




.1 40 


AAA 

.444 




Hales (1976) 


7 
/ 




.194 


.366 




Hales (1980) 


i O 


.OOU 


.o47 


.380 




Hale* (1981 ) 


1 Q 

10 


.477 


.340 


.477 




Hales (1982) 


I <£ 


.04o 


.281 


.548 




Halliwell (1975) 

* •■ill f f w II \ 1 V # w J 


0 


1 no 
. I Uo 


.022 


.108 




farm an /1Q7*J\ 

Jcrrnan \ i y / o/ 


12 


- .210 


.224 


- .210 




V^im AKUaH 9, A LL.u /mill 

i\eim*M0D0tt « Abbott (i977) 


1 


-1.182 




—1.182 




k'ruirilcLru JL QaL-**- /1Q"»C\ 


20 


- .398 


.731 




- .398 


NiflDS 


24 


.370 


.380 


.551 


.189 


Leinnarot « cngei (1981 ; 


2 


.295 


.375 


.295 




1 autn 11 G7Q\ 

Levin (19/9/ 


2 


1.005 


.629 


1.005 




t inn »t al M Q7fil 

Linn ex ai. \ i y /oj 


n 

6 


.741 


.538 


.741 




mirQuez uy/o/ 


8 


.239 


.195 


.300 


.178 


lYicuume o< uertose (lyoi; 


8 


1.022 


.397 


1.060 


.757 


Parr At al MQR1 \ 
r 411 ct 31. v 1 yo 1 / 


1 c 
lb 


- .096 


.231 


- .141 


.198 


Ravder et al (1978) 


A 
*f 


.04/ 


011 
.oil 


.64/ 




Slavin et al. (1983) 


7 


.237 


.099 


.237 




Slavin et al. (in press) 


13 


.477 


.310 


.227 


.492 


Stefanides (1976) 


2 


1.728 


1.069 


.972 


2.484 


Stone & Vaughn (1976) 


2 


1.215 


.081 


1.215 




Wang (1979) 


4 


.849 


.451 




.948 


Wang & Stiles (1976a) 


8 


.691 


.817 






Wang & Stiles (1976 b) 


10 


.738 


.571 




.738 


Wendt (1980) 


4 


.094 


.092 


.094 




Whitley (1979) 


12 


.523 


.250 




.523 



x .445 .383 .394 .601 .694 

S -°- .488 .275 .435 .746 .120 

Note . Total N of studies 3 38 

Total N of comparisons = 309 
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and 50 for the experimental and control groups respectively* The mean 
study-weighted effect size for the 81 comparisons that focused on student 
affective outcomes is .60, while the mean atudy^neighted effect site for the 
20 comparisons that contained behavioral outcomes is .69. These correspond to 
percentiles of 73 for affective outcomes and 75 for behavioral outcomes. 

The relationship of each independent variable to the mean study-weighted 
effect size was tested for significance using ANOVA. Table 2 lists the 
JF-values for the 39 conditioning variables for all 38 studies. The results 
indicate that none of the variables had a statistically significant impact on 
the mean study-weighted effect size. This suggests that no one conditioning 
variable affected the overall student outcomes or the gains attributable to 
the adaptive education program, and that any one produced robust, consistent, 
and ooderately large effecte. In other words, the results do not differ 
significantly across categories of odaptiveness, student outcomes, social 
contexts, methodological rigor and characteristics of the study, and subject 
characteristics. 

Conclusion 

The results of the synthesis show strong and consistent effects of 
adaptive education. The mean effect size of .45, or nearly half a standard 
deviation, is considerably larger than the average of several dozen syntheses 
of productive factors in classroom learning compiled by Walberg (198A). In 
fact, iz is more than twice the average effect size of .20 standard deviations 
in classroom research of the past few decades that has been recently 
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Table 2 



F-Values for the Impact of All Conditioning 
Variables on the Mean Study-Weighted Effect Size* 



Variable 



F- Value 



Adaptiveness Characteristics 

Determination of initial capabilities 
Continuous assessment 
Individualized progress 
Periodic evaluations 
Student self-management 
Alternative routes and materials 
Student choice of goals 
Peer assistance 

Outcomes 
Cognitive 
Affective 
Behavioral 

Methodological Rigor 

Contamination and compensation 
Control method 
Generalizability 
History 

Instrument type 
Maturation 

Method of observation 
Mortality 

Operational irrelevancy 
Pretest equivalency 
Reliability of measures 
Reliability of treatment 
Selection bias 
Statistical power 
Statistics 

Social Contexts 
Community type 
Region 
School type 

Study characteristics 
Publication features 
Sample size 
Subject area 
Unit of analysis 
Year 

Subject Characteristics 
Ability level 
Grade level 
Race 
Sex 
SES 



1.57 
.64 
.00 

1.43 
.80 
.14 
.93 
.81 

.20 
.31 
.36 

.22 
1.65 
.59 
.73 
1.34 
.54 
1.25 
1.77 
1.01 
1.41 
.24 
.73 
1.71 
.32 
.30 

.31 
.17 
.12 

.19 
1.25 
.87 
.64 
1.19 

.21 
.27 
.12 
.60 
.11 
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synthesized. The relatively strong effects are also robust and consistent 
across contexts of learning and categories of students. Such robustness and 
consistency are scientifically valuable because the results are generalized 
across a wide variety of conditions that: have been investigated, as well as 
across grade levels and student characteristics such as ability, socioeconomic 
status, race, and sex. 

In contrast to the results from earlier reports of inconsistent and low 
or negative effects of adaptive instruction, the consistent outcome* reflected 
in the past decade" s reports of research on the effects of adaptive 
instruction are particularly noteworthy. Tailoring instruction to respond to 
the learning characteristics and needs of individual students seems to be an 
educational alternative that can be effective in obtaining intended social and 
academic outcomes. 
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APPENDIX 



Information Coded for Each Study 



Adaptiveness Characteristic? 



Coding Specifications 



Determination of 
initial capabilities 



Unspecified - 0; Inadequately 
specified or implied - 1; 
Adequately specified - 2. 



Continuous assessment 
of capabilities 



Unspecified *» 0; Inadequately 
specified or iaplied - 1; 
Adequately specified * 2. 



Individualized progress 



Unspecified ■ 0; Inadequately 
specified or iaplied - 1; 
Adequately specified - 2. 



Periodic evaluations 
of progress 



Unspecified - 0; Inadequately 
specified or iaplied - 1; 
Adequately specified ■ 2. 



Student self-management 



Unspecified - 0; Inadequately 
specified or iaplied « 1; 
Adequately specified » 2. 



Alternative routes 
and materials 



Unspecified - 0; Inadequately 
specified or iaplied - 1; 
Adequately specified - 2, 



Some student choice 
of goals 



Unspecified - 0; Inadequately 
specified or iaplied - 0; 
Adequately specified - 2. 
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Peer assistance 
and group projects 



Unspecified - 0; Inadequately 
specified or implied - 1; 
Adequately specified - 2. 



Outcomes 



Cognitive outcomes 



Unspecified « 00; Academic 



achievement , factual knowledge - 01; 
Academic achievement, higher level, 
(comprehension, understanding, 
concept attainment) - 02; I.Q. 
tests - 03; Grades m 04; Discrimina- 
tion learning - 05; Accuracy « 06; 
No. of problems attempted - 07; 
Behavior modification - 08; Accuracy 
task-general « 09; Correctly 
completed-general ■ 10; Correctly 
completed task-general - 11; 
Accuracy-reading/writing, English 
- 12. 



- 01; School concept (attitude 
toward school, teacher, etc.) 

- 02; Creativity » 03; Curiosity 

- 04; Independence - 05; 
Adjustment, security - 06; Locus 
of control - 07; Cooperation 

08; Persistence - 09; 

Motivation - 10; General affective 

gain - 11; Teacher attitude * 12. 



Affective outcomes 



Unspecified - 00; Self-concept 



Behavioral outcomes 



Unspecified - 00; Time-on- task 
« 01; Socialization - 02; 
Opportunity to learn - 03. 



9 

ERIC 



A-2 

27 



Methodological Rigor 



Contamination and Do untreated control groups 

compensation imitate or somehow gain the 

benefits of a treatment! 
or work harder to compete 
with experimental group 
(evidence of teacher 
interchange or other bias)? 

Unspecified - 0; Inadequate 
indication - 1; Adequate 
indication » 2. 



Control method Unspecified » 0; Stratification 

(e.g., X y means, Chi square) » 1; 
Partial correlation » 2; Beta 
weights in regression - 3; Raw or 
metric weights in regression » 5; 
Factorial analysis of variance « 
6; Analysis of covariance » 7; 
None - 8. 



Generalizability Can the cause-effect be generalized 

to other times, units, or settings 
(random or typical samples of 
students, classes, or schools)? 

Unspecified » 0; Inadequate 
indication - 1; Adequate 
indication « 2. 



History Have external factors in the 

environment, other than the 
presumed treatment causes, 
brought about the resultant 
change (evidence of 
differential teacher assignment 
or other bias)? 

Unspecified » 0; Inadequate 
indication » 1; Adequate 
indication « 2. 



A-3 



ERLC 



28 



Instrument type 

for dependent variable 



Unspecified » 0; Local « 1; 
Standardised « 2. 



Maturation Have factors within the units, 

rather than the presumed treat- 
ment causes, brought about cMnges 
or differences (extended 
treatment or without random 
assignment to treatments and 
control conditions)? 

Unspecified - 0; Inadequate 
indication - 1; Adequate 
indication » 2. 



Method of observation 
of independent variable 



Unspecified « 0; Student 
questionnaire m 1; Teacher 
report, observer report, 
etc. - 2; Other - 3. 



Mortality Do differential dropout rates in 

the treatment, or control groups, 
account for differences on the 
dependent variable obtained at 
the end of the experiment 
(self-evident)? 

Unspecified « 0; Inadequate 
indication * 1; Adequate 
indication - 2. 



Operational irrelevancy Does the treatment, as administered, 

contain Hawthorne, experimenter- 
bias, placebo, or other unintended 
causes that produce the effects? 

Unspecified * 0; Adequate-self-evident 
in the report - 2. 
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Pretest equivalency 



Have the initial differences 
between the two groups been 
accounted for? 



Unspecified - 0; Statistical 
control (e.g., ANCOVA, regression) 
» 1; Random assignment * 2; 
Statistical control and random 
assignment » 3; Gain scores » A. 



Reliability of measures Are the outcome measurement 

instruments internally 
consistent or stable enough to 
detect changes during, or 
differences in, treatments? 

Unspecified m 00; Adequate « 01; 
Actual reliability figure * NN. 



Reliability of Have the units in a treatment 

treatment actually undergone the same 

conditions as the one in the 
control group? 

Unspecified * 0; Inadequate 
indication » 1; Adequate 
indication ■ 2» 



Selection bias Do pre-existing differences among 

the groups account for later 
differences on the dependent 
variable (e.g., different I.Q. 
levels)? 

Unspecified « 0; Inadequate 
indication » 1; Adequate 
indication » 2, 
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Statistical power 



Is the sample size large enough 
to reject the null hypothesis 
at a given level of probability 
or are estimate coefficients 
with reasonably small margin of 
error (60 or more classes; 100 
or more individuals)? 

Unspecified « 0; Inadequate 
indication « 1; Adequate 
indication » 2. 



Statistics 



Jt-value « 1; F-value ■ 2; 
Correlations « 3; Chi- 
square » 4; Means « 5. 



Social Contexts 



Community type 



Unspecified « 0; Urban « 1; 
Rural M 2; Suburban * 3; 
Other « 4. 



Region 



Unspecified « 0; East « 1; 
Midwest « 2; West « 3; South 
■ 4; More than one « 5; 
Other country ■ 6. 



School type 



Unspecified « 0; Public « 1; 
Private « 2, 



Study Characteristics 
Publication features 



Unpublished - 1; Published - 2; 
Ph.D# dissertation * 3 # 



Sample size 



Unspecified - 0000; 4 spaces 
for exact N. 
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Subject area 



Unspecified « 0; Reading ■» 1; 
Math - 2; Language arts - 3; 
Science - 4; Social studies 
- 5; Other - 6. 



Unit of analysis 



Unspecified - 0; Individual - 2; 
Class - 3; Grade - 4; Pod, module, 
etc. - 5; School - 6; Varied « 7. 



Year of study 



4 spaces for each year. 



Subject Characteristics 



Ability level 



Unspecified - 0; All low - 1; All 
middle - 2; All high - 3; Special 
education students - 4; Mixed/varied 
- 5. 



Grade level 



Unspecified - 00; First grade 

- 01; Second grade - 02; Mixed 
primary (K-3) - 22; Mixed 
middle (4-6) - 23; Mixed upper 
(7-8) - 24; Mixed high school 

- 25; Mixed other « 26. 



Race 



Unspecified « 0; Black * 1; 
White - 2; Latino - 3; 
Oriental * 4; Mixed - 5; Other 
Other ethnic - 6. 



Sex 



Boys =- 1; Girls - 2; Mixed » 3. 



Socioeconomic status 



Unspecified - 0; Lower - 1; 
Middle = 2; Upper - 3; Mixed 
= 4. 
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